Method and apparatus for creating and displaying interactive three dimensional computer images

ABSTRACT

Disclosed is a system and method for creating and displaying interactive three dimensional (3D) computer images. In a preferred embodiment of the system a plurality of digital cameras are positioned in a single arcuate vertical plane, aimed at a center of an object to be imaged in the center of a rotating turntable. Images of the object are captured by each of the cameras after the turntable rotates a certain number of degrees, until the object has rotated a complete 360 degrees. The captured images are then sent to a computer where they are stored and from them, virtual images are created. These original images and virtual images are then compressed and re-sampled in order to be interactive 3D computer images. The 3D images of the object are then displayed and manipulated by the view through using a computer controller, such as a mouse, and various control icons.

PRIORITY

[0001] This application claims priority to a U.S. Provisional Application entitled “Method and Apparatus for Creating Interactive Three Dimensional Computer Images and Display Thereof” filed with the U.S. Patent and Trademark Office on Oct. 16, 2000 and assigned Ser. No. 60/240,746.

BACKGROUND OF THE INVENTION

[0002] 1. Field of the Invention

[0003] The present invention generally relates to computerized graphics, and in particular, to a method and apparatus for creating and manipulating interactive three dimensional (3D) computer images.

[0004] 2. Description of the Related Art

[0005] Early computers dealt primarily with the processing and displaying of text and numbers, such as word processing and mathematical computations. The advances in computer graphics have extended the range of capabilities for the user. Objects can now be displayed in three dimensional (3D) representation, for example in wire frame, solid and/or shaded forms.

[0006] Computer graphics enable simulation, on a display screen, of a displayable object. For example, a computer graphics system enables a user such as a designer to generate an image of a 3D object, such as a chair, for an example. Most computer graphic systems will then allow the user to move the image of the chair around the screen, even permitting the user to turn the chair around, turn the chair upside down, or rotate the chair about any of several possible axes.

[0007] The ability to move a displayed object is advantageous in computerized graphics. Such movement enables the viewer to see the different aspects of the displayed object. Being able to see the graphically displayed object from other perspectives is important, for example, when purchasing an item over the Internet. This enables consumers of items over the Internet to be more informed on their buying decisions. Therefore, a need exists for an efficient and cost effective system and method for creating and displaying 3D images.

SUMMARY OF THE INVENTION

[0008] It is, therefore, an object of the present invention to provide a system and method for creating and displaying three dimensional (3D) computer images.

[0009] It is another object of the present invention to provide interactive 3D computer images that a viewer can manipulate to better visualize the object being imaged.

[0010] To achieve the above objects, there is provided a system and method for creating and displaying interactive 3D computer images. In a preferred embodiment of the system a plurality of digital cameras are positioned in a single arcuate vertical plane, aimed at an object to be imaged in the center of a rotating turntable. Images of the object are captured by each of the cameras after the turntable rotates a certain number of degrees, until the object has rotated a complete 360 degrees. The captured images are then sent to a computer where they are stored and from them, virtual images are created. These original images and virtual images are then compressed and re-sampled in order to be interactive 3D computer images. The 3D images of the object are then displayed and manipulated by the view through using a computer controller, such as a mouse, and various control icons.

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

[0012]FIG. 1 is a diagram illustrating a preferred embodiment of the present invention;

[0013]FIG. 2A illustrates the positioning of the digital cameras when capturing the images of the object;

[0014]FIG. 2B illustrates the positioning of the cameras in relation to the object that is created through the virtual images;

[0015]FIG. 3 is a diagram illustrating an example of a virtual image created from an actual image;

[0016]FIG. 4 is a diagram illustrating sphering an object;

[0017] FIGS. 5A-5C are flow charts illustrating a method of displaying an interactive 3D image according to the present invention; and

[0018]FIG. 6 is an example of a display with a magnifying window open.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0019] A preferred embodiment of the present invention will be described herein below with reference to accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

[0020] In the present invention, a preferred method for creating and displaying interactive three dimensional computer images comprises the steps of: studio processing an object; computer processing images; sphering the object; image re-sampling and compression; and interactively displaying the three dimensional image. The preferred embodiment and method of the invention will be described herein below.

[0021] I. Studio Processing an Object

[0022]FIG. 1 is a diagram illustrating a preferred embodiment of the present invention. In FIG. 1, a plurality of cameras (in this case four digital cameras) A, B, C, and D are fixed in a single arcuate vertical plane, spaced apart from each other, preferably 20 degrees apart. In the preferred embodiment, camera A is fixed at an angle of 20 degrees from the horizontal. As a result, camera B is position at 40 degrees, camera C is positioned at 60 degrees, and camera D is positioned at 80 degrees. In an alternative embodiment, the cameras may be place in a same plane not necessarily vertical, for example, at 45 degrees. The four cameras A-D are aimed at the center of a turntable 100, which is rotatable 360 degrees. An object 50 to be studio processed is placed in the center of the turntable 100 to be photographed. Once the object 50 is centered, cameras A-D each capture a first image of the object 50 with the turntable 100 stationary, i.e. rotated 0 degrees with respect to a reference marker “X”. These images are then labeled A1, B1, C1, and D1 based on which camera captured the image. The turntable 100 is then rotated 30 degrees, and the cameras A-D each capture a second image of the object 50. These images are then labeled A2, B2, C2, and D2. This image capturing process is repeated in predetermined increments, in this case, 30 degree increments, until the turn table 100 has rotated 360 degrees and each of the cameras A-D have taken 12 images of the object 50 for a total of 48 images.

[0023] Each of the cameras A-D are connected to a computer 104 including a frame grabber 106 for controlling the cameras A-D. The frame grabber 106 activates each of the cameras A-D and stores the captured images in the memory of computer 104. As described above, preferably each camera will take twelve images. Camera A's images are labeled A1-A12, camera B's images are labeled B1-B12 and cameras C and D's images are labeled the same way. When the studio processing of the object 50 is complete, there are a total of forty-eight images of object 50 stored in the computer 104.

[0024] It should be noted, however, that although in the preferred embodiment described above, the turntable 100 is rotated 360 degrees and 12 images are taken of the object, it is also possible to vary the amount the turntable 100 is rotated and the number of images taken of the object.

[0025] II. Computer Processing the Images

[0026] After the studio processing is completed, forty-eight images (A1-D12) of object 50 are stored in the computer 104. From these forty-eight images, computer software is used to create forty-eight virtual images of object 50. These virtual images of object 50 are computer generated views of the object from different perspectives. To further explain, the 48 actual images are views of object 50 from approximately 0 to 90 degrees on an arcuate vertical plane. However, in order to create an interactive 3D view of object 50, the views of the object from approximately 90 to 180 degrees are necessary. The forty-eight virtual images created from the actual images of object 50 are the views of object 50 from approximately 90 to 180 degrees. To create virtual images the computer rotates the actual image 180° or flips and reverses the actual image. FIGS. 2A, 2B and 3 are diagrams illustrating the relationship of the virtual images generated by the computer 104 compared to the actual images taken of object 50.

[0027]FIG. 2A is a diagram illustrating the positioning of the digital cameras when capturing the images of the object. As illustrated in FIG. 2A, camera A captures an images of object 50 from 20 degrees. Cameras B-D take image of object 50 along the same vertical plane as camera A, from 40 degrees to 80 degrees. These four images (A1, B1, C1, and D1) are then used to simulate a single camera moving up and over the object 50 in the same vertical plane. However, there are no cameras at 100 degrees, 120 degrees, 140 degrees, and 160 degrees to capture the images of object 50 from these perspectives that are necessary in order to provide an interactive 3D image. Therefore, the virtual images are created from the actual images taken of object 50 to simulate the images that cameras at 100 degrees, 120 degrees, 140 degrees and 160 degrees would capture.

[0028]FIG. 2B illustrates the positioning of the virtual cameras in relation to the object that is created through the virtual images. In FIG. 2B, virtual cameras E-H are presented to better illustrate what is represented by the virtual images (i.e. the images of object 50 from approximately 90 to 180 degrees in a single vertical plane). Therefore, by 483-6 combining the actual images with the virtual images, a 3D image is created in which, a user can rotate the image from 0 degrees, up and over the object, to 180 degrees.

[0029] It should be noted, however, that the virtual images E1-H1 are not created from actual images A1-D1, but from A7-D7. This is because E1-H1 are the virtual images of the opposite side of object 50 from images A1-D1 and images A7-D7 are taken when the turn table 100 has been rotated 180 degrees, thereby exposing the opposite side of object 50 to cameras A-D. The actual images A1-D1 are used to create the virtual images E7-H7. An example of a virtual image created from an actual image is illustrated in FIG. 3. As a result, the eight images, both actual and virtual, making up the 0 to 180 degree view of object 50 along the single vertical plane are A1, B1, C1, D1, E1, F1, G1, and H1 or coming from the other direction A7, B7, C7, D7, E7, F7, G7, and H7. Tables 1-4 below illustrate the detailed manner in which the virtual images are created from actual images.

[0030] It should noted that the computer processing “flip & reverse” can also be “reverse & flip”, or any other image processing techniques that achieves the same or similar results may be used. Also, in the computer image process rotation, the degree rotated may be varied to achieve different viewing effects. Further, the process of creating the virtual images may also be performed either before or after image compression and re-sampling, which will be described here in below.

[0031] Tables 1-4 provided herein below, further illustrate the relationship between the actual images and the virtual images. TABLE 1 Computer Image Processing for Virtual Images E1-E12 Actual Image Actual Virtual Image from camera D, Image from Virtual which is at a with Turn Camera E, which Virtual Image 80° viewing Table Computer Image is at a 100° with the turn plane. rotated: Processing viewing plane. table rotated: D7 180° Rotate D7 by 180° E1 180° + 180° = 0° (or flip & reverse) D8 210° Rotate D8 by 180° E2 210° + 180° = 30° (or flip & reverse) D9 240° Rotate D9 by 180° E3 240° + 180° = 60° (or flip & reverse) D10  270° Rotate D10 by 180° E4 270° + 180° = 90° (or flip & reverse) D11  300° Rotate D11 by 180° E5 300° + 180° = 120° (or flip & reverse) D12  330° Rotate D12 by 180° E6 330° + 180° = 150° (or flip & reverse) D1  0° Rotate D1 by 180° E7  0° + 180° = 180° (or flip & reverse) D2  30° Rotate D2 by 180° E8  30° + 180° = 210° (or flip & reverse) D3  60° Rotate D3 by 180° E9  60° + 180° = 240° (or flip & reverse) D4  90° Rotate D4 by 180° E10   90° + 180° = 270° (or flip & reverse) D5 120° Rotate D5 by 180° E11  120° + 180° = 300° (or flip & reverse) D6 150° Rotate D6 by 180° E12  150° + 180° = 330° (or flip & reverse)

[0032] TABLE 2 Computer Image Processing for Virtual Images F1-F12 Actual Image Actual Virtual Image from camera C, Image from Virtual which is at a with Turn Camera F, which Virtual Image 60° viewing Table Computer Image is at a 120° with the turn plane. rotated: Processing viewing plane. table rotated: C7 180° Rotate C7 by 180° F1 180° + 180° = 0° (or flip & reverse) C8 210° Rotate C8 by 180° F2 210° + 180° = 30° (or flip & reverse) C9 240° Rotate C9 by 180° F3 240° + 180° = 60° (or flip & reverse) C10 270° Rotate C10 by 180° F4 270° + 180° = 90° (or flip & reverse) C11 300° Rotate C11 by 180° F5 300° + 180° = 420° (or flip & reverse) C12  330° Rotate C12 by 180° F6 330° + 180° = 450° (or flip & reverse) C1  0° Rotate C1 by 180° F7  0° + 180° = 180° (or flip & reverse) C2  30° Rotate C2 by 180° F8  30° + 180° = 210° (or flip & reverse) C3  60° Rotate C3 by 180° F9  60° + 180° = 240° (or flip & reverse) C4  90° Rotate C4 by 180° F10   90° + 180° = 270° (or flip & reverse) C5 120° Rotate C5 by 180° F11  120° + 180° = 300° (or flip & reverse) C6 150° Rotate C6 by 180° F12  150° + 180° = 330° (or flip & reverse)

[0033] TABLE 3 Computer Image Processing for Virtual Images G1-G12 Actual Image Actual Virtual Image from camera B, Image from Virtual which is at a with Turn Camera G, which Virtual Image 40° viewing Table Computer Image is at a 140° with the turn plane. rotated: Processing viewing plane. table rotated: B7 180° Rotate B7 by 180° G1 180° + 180° = 0° (or flip & reverse) B8 210° Rotate B8 by 180° G2 210° + 180° = 30° (or flip & reverse) B9 240° Rotate B9 by 180° G3 240° + 180° = 60° (or flip & reverse) B10  270° Rotate B10 by 180° G4 270° + 180° = 90° (or flip & reverse) B11  300° Rotate B11 by 180° G5 300° + 180° = 120° (or flip & reverse) B12  330° Rotate B12 by 180° G6 330° + 180° = 150° (or flip & reverse) B1  0° Rotate B1 by 180° G7  0° + 180° = 180° (or flip & reverse) B2  30° Rotate B2 by 180° G8  30° + 180° = 210° (or flip & reverse) B3  60° Rotate B3 by 180° G9  60° + 180° = 240° (or flip & reverse) B4  90° Rotate B4 by 180° G10   90° + 180° = 270° (or flip & reverse) B5 120° Rotate B5 by 180° G11  120° + 180° = 300° (or flip & reverse) B6 150° Rotate B6 by 180° G12 150° + 180° = 330° (or flip & reverse)

[0034] TABLE 2 Computer Image Processing for Virtual Images H1-H12 Actual Image Actual Virtual Image from camera A, Image from Virtual which is at a with Turn Camera H, which Virtual Image 20° viewing Table Computer Image is at a 160° with the turn plane. rotated: Processing viewing plane. table rotated: A7 180° Rotate A7 by 180° H1 180° + 180° = 0° (or flip & reverse) A8 210° Rotate A8 by 180° H2 210° + 180° = 30° (or flip & reverse) A9 240° Rotate A9 by 180° H3 240° + 180° = 60° (or flip & reverse) A10  270° Rotate A10 by 180° H4 270° + 180° = 90° (or flip & reverse) A11  300° Rotate A11 by 180° H5 300° + 180° = 120° (or flip & reverse) A12  330° Rotate A12 by 180° H6 330° + 180° = 150° or flip & reverse) A1  0° Rotate A1 by 180° H7  0° + 180° = 180° (or flip & reverse) A2  30° Rotate A2 by 180° H8  30° + 180° = 210° (or flip & reverse) A3  60° Rotate A3 by 180° H9  60° + 180° = 240° (or flip & reverse) A4  90° Rotate A4 by 180° H10   90° + 180° = 270° (or flip & reverse) A5 120° Rotate A5 by 180° H11  120° + 180° = 300° (or flip & reverse) A6 150° Rotate A6 by 180° H12  150° + 180° = 330° (or flip & reverse)

[0035] III. Sphering the Object

[0036] The images of the object captured in the two above steps, both the actual images and the virtual images, enable creating an interactive 3D display in the shape of a semi-sphere of the object 50. In order to sphere the object, the object 50 is placed in the center of the turntable 100 upside down and the inverted object 50 is studio processed and computer image processed by the same apparatus as illustrated in FIG. 1. The result is an interactive 3D display in the shape of a semi-sphere of the inverted object 50. FIG. 4 is a diagram illustrating sphering the object. By combining the two sets of semi-sphere images, a 360° interactive 3D image of the object is created. In other words, an interactive 3D display in the shape of a sphere of the object is created.

[0037] IV. Image Compression and Re-Sampling

[0038] After the images of the object are taken and processed, the next step is compressing the images to reduce the memory required to display the images and re-sampling the images in different sizes to interactively display them. In the preferred embodiment of the present invention, the images are compressed and re-sampled into four different sizes of compressed images. The compressed images are then stored, respectively, as B (Best View Images), S (Small Size Images), M (Medium Size Images), and L (Large Size Images). A user then has the choice of displaying the different sized images of the object 50.

[0039] V. Interactively Displaying the 3D Image

[0040] After completing steps I-IV illustrated above, it is possible to display the interactive 3D image with the re-sampled compressed images. The present invention is applicable to any type of electronic visual display but for simplicity, will be described herein below with reference to a web page display on a computer.

[0041] FIGS. 5A-5C are flow charts illustrating a method of displaying an interactive 3D image according to the present invention. In FIG. 5A, when a viewer logs on to a user web site utilizing interactive 3D image displays of the present invention, a program code such as a JAVA applet for the interactive 3D image is loaded in step 500. It is then determined whether the viewer has a license to view the web site in step 502. If yes, the viewer is directed to select which size image to view in step 506, and if not, the viewer is redirected from the site to a message indicating why access is denied in step 504.

[0042] If the viewer selects the smaller image “S” in step 506, the S image is loaded and displayed in step 508. At this point, in step 512, the viewer has the option of enlarging the image by left clicking a mouse over the image or selecting a respective icon and loading a larger image. If the image size is acceptable to the viewer in step 512, the viewer can interactively manipulate the image (rotate the image horizontally or vertically) in steps 514 and 518. If the viewer chooses to manipulate the image in steps 514 and 518, for example, by either right clicking or left clicking a mouse and dragging, the object is manipulated in steps 516 and 520 respectively. If in step 514 the viewer left clicks and drags the mouse, the image is rotated horizontally in step 516. If the viewer right clicks and drags in step 518, the image is rotated vertically in step 520. When the user has finished rotating, the process returns to step 508.

[0043] If the viewer had chosen the larger image in step 506, moving the process to step 510, or decided to enlarge the image in step 512, the process then moves to step 522 in FIG. 5B. In step 522, the larger image (medium sized image “M”) is loaded and rotates automatically in the horizontal direction. By left clicking the mouse on the image in step 524 the image stops rotating and is displayed as a still image in step 526. By left clicking on the still image again in step 528 the image continues rotating horizontally, returning to step 522. In step 530, if the viewer left clicks and drags or positions the mouse pointer on the side borders of the image, the viewer controls horizontal rotation in step 532. In step 534, if the viewer right clicks and drags or positions the mouse pointer on the top or bottom borders of the image, the viewer controls vertical rotation of the image in step 536. In step 538, if the viewer places the mouse pointer in any of the four comers of the image, the image is rotated in the diagonal direction in step 540. After either step 532, 536, or 540, when the user stops rotating the object in the respective direction, the process returns to step 522.

[0044]FIG. 5C is a continuation of the flow chart illustrated in FIGS. 5A and 5B, and further demonstrates the possible manipulations of the displayed image. In FIG. 5C, the viewer can select the best view “B” image of the displayed object in step 542. If the user selects the best view icon, the B image is loaded in a new window as a still image in step 544. In a more advanced wider bandwidth network environment, all the manipulations performed on the images can be performed on the best view B image.

[0045] If the best view is not selected in step 542, the viewer can select the magnifying icon in step 548. When selected, a moveable viewing window opens over the image and the large “L” images are displayed in the window in step 550. The L sized images displayed in the magnifying window are magnified images of the object being displayed relative to the windows positioned over the full view M sized image. FIG. 6 is an example of a display with the magnifying window opened. In the magnifying window mode, the viewer can left click to zoom in further, and right click to zoom back out in step 552. By clicking on the magnifying icon again, the magnifying window is closed in step 554 and the images returns to automatic rotation in step 522 or a still mode in step 526.

[0046] The viewer has the option of speeding up or slowing down the automatic rotation in steps 556 and 560, by selecting a plus or minus icon displayed with the image. If in step 556 the user chooses to speed up the rotation of the image, each individual image is displayed for a shorter amount of time in step 558. If in step 560, the user chooses to slow down the rotation of the image, each individual image is displayed for a longer amount of time. Also, in step 564, the viewer may select viewing the image in a music mode by selecting the music icon that turns the sound on and off in step 566.

[0047] After all the image manipulations described in steps 558, 562, 564, and 566, when user is finished, the process returns to step 522 or step 526.

[0048] While a preferred embodiment of the present invention has been described herein above, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. For example, in the step of studio processing the object a different number of cameras could be used, the angles between the cameras can be different, the number of images taken can be changed, and different types of camera could be used. Instead of using digital cameras capturing still images, video cameras could be used or film type cameras could be used and the developed images could then be loaded into the computer using a scanner, by way of example.

[0049] When compressing and re-sampling the images, the order of compressing and re-sampling can be varied, as well as the number re-sized images, i.e. six different sizes. Also, the images can be compressed using any number of methods. 

What is claimed is:
 1. A method for creating and displaying interactive three dimensional (3D) computer images, the method comprising: studio processing an object into images; computer processing the images to create virtual images; compressing and re-sampling the computer processed images for interactive 3D display; and displaying the interactive 3D images.
 2. A method for creating and displaying interactive 3D computer images as recited in claim 1, further comprising sphering the interactive 3D images.
 3. A method for creating and displaying interactive 3D computer images as recited in claim 1, wherein the step of studio processing the object into images comprises: centering the object on a turntable; positioning a plurality of cameras in a same arcuate plane aimed at the object; and photographing the object with each camera a plurality of times after incrementally rotating the turntable a predetermined distance for each photograph.
 4. A method for creating and displaying interactive 3D computer images as recited in claim 3, wherein the step of positioning a plurality of cameras in a same vertical plane aimed at the object comprises: positioning a first camera at twenty degrees from a horizontal even with a top surface of the turntable; positioning a second camera at twenty degrees from the first camera; positioning a third camera at twenty degrees from the second camera; and positioning a fourth camera at twenty degrees from the third camera.
 5. A method for creating and displaying interactive 3D computer images as recited in claim 1, wherein the step of computer processing the images to create virtual images comprises utilizing computer software to create from the images, the virtual images of the object viewed from different perspectives.
 6. A method for creating and displaying interactive 3D computer images as recited in claim 2, wherein the step of sphering the interactive 3D images comprises: inverting and centering the object on the turntable; studio processing the inverted object into inverted images; computer processing the inverted images to create virtual inverted images; and combining the computer processed images of the object with the computer processed images of the inverted object to create a 360 degree spherical view of the object.
 7. A method for creating and displaying interactive 3D computer images as recited in claim 1, wherein the step of compressing and re-sampling the computer processed images for interactive 3D display comprises compressing and re-sampling the computer processed images to create a plurality of views of the images for interactive 3D display on a computer display.
 8. A method for creating and displaying interactive 3D computer images as recited in claim 7, wherein a plurality of views of the images includes a best view and different sizes.
 9. A method for creating and displaying interactive 3D computer images as recited in claim 1, wherein the step of displaying the interactive 3D images comprises: loading a program code for the displaying the interactive 3D images; selecting a 3D image to display; displaying the 3D image; diagonally rotating the 3D image; horizontally rotating the 3D image; and vertically rotating the 3D image.
 10. A method for creating and displaying interactive 3D computer images as recited in claim 9, wherein the step of selecting the 3D image to display further comprises selecting a best view.
 11. A method for creating and displaying interactive 3D computer images as recited in claim 9, wherein horizontally rotating the 3D image is performed automatically.
 12. A method for creating and displaying interactive 3D computer images as recited in claim 11, wherein the step of horizontally rotating the 3D image automatically, further comprises controlling a rotation speed of the 3D image.
 13. A method for creating and displaying interactive 3D computer images as recited in claim 9, wherein the step of vertically rotating the 3D image is performed automatically.
 14. A method for creating and displaying interactive 3D computer images as recited in claim 13, wherein the step of vertically rotating the 3D image automatically, further comprises controlling a rotation speed of the 3D image.
 15. A method for creating and displaying interactive 3D computer images as recited in claim 9, wherein the step of diagonally rotating the 3D image is performed automatically.
 16. A method for creating and displaying interactive 3D computer images as recited in claim 15, wherein the step of diagonally rotating the 3D image automatically, further comprises controlling a rotation speed of the 3D image.
 17. A method for creating and displaying interactive 3D computer images as recited in claim 9, wherein the step of displaying the interactive 3D images, further comprises magnifying an area of the 3D image by selecting a magnifying control icon and positioning a window over the area of the 3D image to be magnified.
 18. A system for creating and displaying interactive three dimensional (3D) computer images, the system comprising: a turntable for rotating an object; a plurality of cameras arranged in a same plane aimed at the object, for photographing the object; and a computer for controlling the cameras, rotating the object, computer processing images of the object, re-sampling and compressing the images and displaying the interactive 3D computer images. 